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Preface  &  Acknowledgements 


Welcome  to  our  Tenth  Annual  Acquisition  Research  Symposium!  We  regret  that  this 
year  it  will  be  a  “paper  only”  event.  The  double  whammy  of  sequestration  and  a  continuing 
resolution,  with  the  attendant  restrictions  on  travel  and  conferences,  created  too  much 
uncertainty  to  properly  stage  the  event.  We  will  miss  the  dialogue  with  our  acquisition 
colleagues  and  the  opportunity  for  all  our  researchers  to  present  their  work.  However,  we 
intend  to  simulate  the  symposium  as  best  we  can,  and  these  Proceedings  present  an 
opportunity  for  the  papers  to  be  published  just  as  if  they  had  been  delivered.  In  any  case,  we 
will  have  a  rich  store  of  papers  to  draw  from  for  next  year’s  event  scheduled  for  May  14-15, 
2014! 


Despite  these  temporary  setbacks,  our  Acquisition  Research  Program  (ARP)  here  at 
the  Naval  Postgraduate  School  (NPS)  continues  at  a  normal  pace.  Since  the  ARP’s 
founding  in  2003,  over  1,200  original  research  reports  have  been  added  to  the  acquisition 
body  of  knowledge.  We  continue  to  add  to  that  library,  located  online  at 
www.acquisitionresearch.net,  at  a  rate  of  roughly  140  reports  per  year.  This  activity  has 
engaged  researchers  at  over  70  universities  and  other  institutions,  greatly  enhancing  the 
diversity  of  thought  brought  to  bear  on  the  business  activities  of  the  DoD. 

We  generate  this  level  of  activity  in  three  ways.  First,  we  solicit  research  topics  from 
academia  and  other  institutions  through  an  annual  Broad  Agency  Announcement, 
sponsored  by  the  USD(AT&L).  Second,  we  issue  an  annual  internal  call  for  proposals  to 
seek  NPS  faculty  research  supporting  the  interests  of  our  program  sponsors.  Finally,  we 
serve  as  a  “broker”  to  market  specific  research  topics  identified  by  our  sponsors  to  NPS 
graduate  students.  This  three-pronged  approach  provides  for  a  rich  and  broad  diversity  of 
scholarly  rigor  mixed  with  a  good  blend  of  practitioner  experience  in  the  field  of  acquisition. 
We  are  grateful  to  those  of  you  who  have  contributed  to  our  research  program  in  the  past 
and  encourage  your  future  participation. 

Unfortunately,  what  will  be  missing  this  year  is  the  active  participation  and 
networking  that  has  been  the  hallmark  of  previous  symposia.  By  purposely  limiting 
attendance  to  350  people,  we  encourage  just  that.  This  forum  remains  unique  in  its  effort  to 
bring  scholars  and  practitioners  together  around  acquisition  research  that  is  both  relevant  in 
application  and  rigorous  in  method.  It  provides  the  opportunity  to  interact  with  many  top  DoD 
acquisition  officials  and  acquisition  researchers.  We  encourage  dialogue  both  in  the  formal 
panel  sessions  and  in  the  many  opportunities  we  make  available  at  meals,  breaks,  and  the 
day-ending  socials.  Many  of  our  researchers  use  these  occasions  to  establish  new  teaming 
arrangements  for  future  research  work.  Despite  the  fact  that  we  will  not  be  gathered 
together  to  reap  the  above-listed  benefits,  the  ARP  will  endeavor  to  stimulate  this  dialogue 
through  various  means  throughout  the  year  as  we  interact  with  our  researchers  and  DoD 
officials. 

Affordability  remains  a  major  focus  in  the  DoD  acquisition  world  and  will  no  doubt  get 
even  more  attention  as  the  sequestration  outcomes  unfold.  It  is  a  central  tenet  of  the  DoD’s 
Better  Buying  Power  initiatives,  which  continue  to  evolve  as  the  DoD  finds  which  of  them 
work  and  which  do  not.  This  suggests  that  research  with  a  focus  on  affordability  will  be  of 
great  interest  to  the  DoD  leadership  in  the  year  to  come.  Whether  you’re  a  practitioner  or 
scholar,  we  invite  you  to  participate  in  that  research. 

We  gratefully  acknowledge  the  ongoing  support  and  leadership  of  our  sponsors, 
whose  foresight  and  vision  have  assured  the  continuing  success  of  the  ARP: 
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Abstract 

Budget  reductions  will  require  the  Department  of  Defense  (DoD)  to  make  difficult  decisions 
on  how  to  invest  limited  resources  and  make  current  programs  more  affordable.  Traditional 
acquisition  methods  are  lengthy,  serial,  gate-like  processes,  built  around  stringent 
specifications  and  arms-length  relationships.  By  contrast,  Challenge-Based  Acquisition 
(ChBA)  utilizes  transparent,  accessible,  concrete  challenges  to  satisfy  warfighter  needs  and 
stimulate  industry  innovation.  Challenges  enable  DoD  programs  to  assess  actual 
performance  against  clearly  defined  mission  objectives  and  create  incentives  for  industry  to 
innovate.  ChBA  thus  offers  a  more  transparent  approach  to  fielding  new  capabilities, 
upgrades,  and  enhancements  to  existing  systems. 

Mandate  for  Change 

It’s  time  to  fundamentally  change  the  way  that  we  do  business  in  Washington.  To 
help  build  a  new  foundation  for  the  21st  century,  we  need  to  reform  our  government  so  that 
it  is  more  efficient,  more  transparent,  and  more  creative.  That  will  demand  new  thinking  and 

a  new  sense  of  responsibility  for  every  dollar  that  is  spent. 

-  President  Barack  Obama  (2009) 

Fewer  than  half  of  the  programs  in  the  Department  of  Defense  (DoD)  Major  Defense 
Acquisition  portfolio  have  met  established  metrics  for  cost  or  performance  (GAO,  2011a). 
Even  worse,  the  DoD  has  canceled  entire  programs  for  cost  overruns  under  the  Nunn- 
McCurdy  Amendment  after  investing  billions  of  dollars  that  could  have  been  used  elsewhere 
across  the  department  (GAO,  2011b).  According  to  the  Government  Accountability  Office 
(GAO),  50  of  74  breaches  involved  engineering  design  issues  discovered  after  production 
had  begun. 

Traditional  DoD  acquisition  follows  a  lengthy,  serial  process  based  upon  a  plethora 
of  documentation  as  required  by  the  DoD  5000  Series  of  Instructions  and  Directives  (DoD, 
2003b)  as  well  as  numerous  Service-specific  acquisition  guidelines.  In  these  documents, 
mission  needs  become  program  requirements,  which  are  then  quantified  as  performance 
parameters,  defined  as  system  attributes,  tracked  through  derived  technical  performance 
measures,  and  included  in  a  government/industry  exchange  of  system  specifications.  Along 
this  serial  path,  the  linkage  of  program  requirements  to  mission  performance  typically 
becomes  unclear  and  often  inaccurate.  Alternatively,  in  some  cases,  system  specifications 
become  far  too  rigid  and  detailed,  thus  stifling  opportunities  for  innovation.  Despite  best 
efforts  by  programs  to  mitigate  risk  through  verification  and  validation  using  the  systems 
engineering  process,  even  a  perfectly  executed  program  can  still  produce  a  quality  product 
that  is  often  “late  to  the  fight,”  operationally  ineffective,  or  unsuitable  even  if  it  addresses  the 
original  mission  need. 

Furthermore,  most  contracts  are  awarded  using  government  source  selection 
evaluations  based  on  industry  paper  proposals  rather  than  “actual”  product  performance. 
This  creates  an  incentive  for  industry  to  produce  flawless  documents  with  highly  optimistic 
cost,  schedule,  and  performance  projections  that  meet  or  exceed  every  requirement  in  the 
government’s  request.  As  a  result,  performance  during  program  execution  often  falls  short  of 
the  government’s  expectations  and  cost  and  schedule  overruns  become  nearly  inevitable. 
These  unrealistic  proposals  become  particularly  problematic  when  there  is  little  prospect  for 
additional  competition  throughout  the  acquisition  life  cycle,  which  may  lock  the  program  into 
a  single  solution  and  provider. 

The  resulting  disappointment  creates  an  arms-length  relationship  between  the 
contractor  and  the  government,  limiting  trust,  communication,  and  transparency.  This  can  be 
particularly  problematic  given  the  long  life  cycle  of  many  defense  acquisition  programs.  The 
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impact  of  this  tense  relationship  can  raise  costs  related  to  bidding  and  negotiating  contracts 
and  slow  the  process  of  coming  to  acceptable  terms  and  conditions  (Crook,  Ketchen, 

Combs,  &  Patterson,  2012).  For  example,  a  recent  study  concluded  that  the  DoD  currently 
spends  roughly  $400  billion  each  year  acquiring  products  and  services  from  its  contractors, 
with  about  $100  billion  of  that  amount  spent  on  administrative  costs  alone.  By  cutting 
unneeded  bureaucracy,  defense  officials  could  reduce  the  department’s  costs  by  20% — or 
roughly  $20  billion  each  year  (Weigelt,  2012). 

The  complexity  of  traditional  DoD  acquisition  makes  the  process  difficult  for 
programs  with  tight  budgets  or  timelines  to  execute  predictably,  and  virtually  impossible  to 
execute  when  trying  to  meet  an  urgent  operational  need.  Given  this  situation,  how  can  the 
DoD  acquire  capabilities  both  faster  and  better?  The  answer  includes  expressing 
requirements  in  terms  of  general  capabilities  rather  than  firm  specifications  and  encouraging 
industry  to  respond  with  applicable  product  development  and  innovation  that  demonstrates 
best-of-breed  solutions. 

This  paper  suggests  Challenge-based  Acquisition  (ChBA)  as  an  approach  that  could 
be  applied  to  urgent  need  situations,  could  be  executed  in  a  more  rapid,  transparent 
manner,  and  would  allow  program  stakeholders  to  satisfy  mission  needs.  ChBA  presents 
challenges  to  a  set  of  interested  parties,  communicates  government  needs  to  the  private 
sector,  and  encourages  the  creation  of  innovative  products.  The  challenges  are  expressed 
in  terms  of  specific  capability  criteria  that  must  be  satisfied,  with  the  proposed  solutions 
proven  by  evidence  of  performance.  The  ChBA  approach  leverages  practices  designed  for  a 
rapidly  evolving  technology  environment  and  meets  the  real  demands  of  users  in  the  field.  It 
applies  acquisition  practices  and  techniques  necessary  to  achieve  better  outcomes  in  DoD 
programs  and  projects.  ChBA  is  founded  on  the  codification  of  government  needs 
expressed  as  concrete  performance  outcomes.  These  outcomes  are  challenges  that  are 
issued  to  a  marketplace  of  competing  vendors,  rather  than  needs  expressed  in  paper 
specification  documents  that  are  addressed  with  unproven  paper  proposals. 

Background 

End  users  have  difficulty  imagining  transformational  or  inventive  solutions  when  they 
have  a  working  solution  at  hand.  Soldiers,  for  example,  are  good  at  improvising  solutions  to 
address  shortcomings  of  equipment,  and  using  whatever  they  can  find  on  the  battlefield. 
Similarly,  they  are  experts  at  assessing  the  likely  success  of  incremental  improvements  to 
devices  and  techniques.  It  is  hard,  however,  to  extend  this  innovation  beyond  the  readily 
conceivable. 

Henry  Ford  supposedly  said,  “If  I  had  asked  people  what  they  wanted,  they  would 
have  said  ‘faster  horses’”  (Ford,  2006).  More  recently,  Steve  Jobs  said,  “You  can’t  just  ask 
customers  what  they  want  and  then  try  to  give  that  to  them.  By  the  time  you  get  it  built, 
they’ll  want  something  new”  (Burlingham  &  Gendron,  1989).  Even  the  brightest  equestrians 
would  have  had  trouble  picturing  the  utility  of  the  Model  T.  While  soldiers,  sailors,  and 
airmen  are  indeed  the  right  individuals  to  define  mission  requirements,  involving  them  in  the 
specification  process  can  limit  the  inventiveness  of  potential  solutions. 

But  suppose  that  Henry  Ford  had  heard,  “I  want  to  get  to  my  destination  faster  and 
with  comfort  and  affordability.”  In  this  case,  the  users  would  have  issued  a  concrete  mission 
challenge — get  there  faster  with  comfort — rather  than  a  specified  solution — a  faster  horse. 
Unfortunately,  government  acquisition  agents,  like  Ford’s  public,  rarely  think  in  terms  of 
mission  challenges  and  instead  think  in  terms  of  tighter  specifications  to  define  solutions. 

As  early  as  the  1980s,  the  DoD  recognized  that  relying  on  highly  rigid  specifications 
can  be  burdensome  and  costly.  Even  in  the  unusual  cases  where  specifications  and 
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standards  are  perfect,  premature  application,  over-application,  and  inappropriate  application 
of  standards  could  still  cause  complex  problems  (Bergman,  1996,  p.  32).  The  DoD  enacted 
acquisition  reforms,  deleting  many  military  specifications  from  contracts,  and  emphasizing 
outcome  and  performance-based  acquisitions  (Bergman,  1996). 

Challenges  present  an  option  for  achieving  these  goals.  Governments  and  industry 
have  long  used  challenges  to  spur  technology  advances  in  areas  that  include  agriculture, 
aviation,  energy,  medicine,  and  navigation.  For  example,  in  1714,  an  Act  of  Parliament 
established  the  British  Longitude  Prize  (Princeton  University,  n.d.).  The  Longitude  Board, 
which  administered  the  prize,  did  not  fund  technical  research  but  simply  promised  monetary 
awards  based  on  the  accuracy  of  proven  results:  £10,000  for  60  nautical  miles  of  accuracy, 
£15,000  for  40  nautical  miles,  and  £20,000  for  30  nautical  miles.  The  prize  prompted 
development  of  the  maritime  chronometer,  which  revolutionized  global  navigation  and 
solved  a  problem  that  had  bedeviled  seafaring  nations  for  over  150  years. 

The  Wright  Brothers’  contract  with  the  U.  S.  Army  (Smithsonian,  National  Air  and 
Space  Museum,  n.d.)  serves  as  a  20th  century  example  of  ChBA.  As  a  result  of  their 
airplane’s  performance  in  the  1909  U.S.  Army  flight  trials,  they  received  a  contract  that 
strongly  incentivized  speed,  with  a  10%  bonus  for  every  full  mile  per  hour  above  40.  The 
average  speed  of  the  Wrights’  aircraft  was  42.5  miles  per  hour,  earning  the  inventors  a 
$5,000  bonus  and  bringing  the  final  purchase  price  of  the  airplane  to  $30,000. 

For  decades,  the  aviation  industry  continued  to  create  ChBA-like  opportunities. 

When  aircraft  operators  abstracted  away  the  details  of  engine  design  and  simply  challenged 
power  plant  makers  to  deliver  performance  in  terms  of  thrust,  weight,  and  efficiency,  General 
Electric’s  Jack  Welch  conceived  the  idea  of  performance-based  logistics.  He  sold  “power  by 
the  hour”  (Knowledge@Wharton,  2007),  which  relieved  aircraft  owners  of  the  need  to 
inventory,  maintain,  and  repair  engines.  As  a  result,  the  costs  of  engine  inventories, 
maintenance,  and  repair  declined  dramatically. 

More  recently,  the  defense  and  aerospace  industries  have  used  challenges  to 
support  innovative  technology  development  in  areas  of  information  technology  (IT),  space 
transportation,  and  military  combat  systems,  as  illustrated  by  the  following  examples. 

Space  Transportation.  In  2004  Space  Ship  One,  a  suborbital  air-launched  space 
plane,  won  the  U.S.  $10  million  Ansari  X  Prize  by  completing  the  first  manned  private  space 
flight.  Space  Exploration  Technologies  Corporation,  also  known  as  SpaceX,  made  history  on 
May  25,  2012,  as  the  world's  first  privately  held  company  to  send  a  cargo  payload,  carried 
on  the  Dragon  spacecraft,  to  the  International  Space  Station  (SpaceX  Corporation,  n.d.). 

Military  Combat  Systems.  Mine  Resistant  Ambush  Protected  (MRAP)  vehicles  are 
a  family  of  armored  fighting  vehicles  originally  designed  under  the  guidance  of  the  U.S. 
Marine  Corps  to  survive  attacks  and  ambushes  involving  improvised  explosive  devices 
(lEDs).  On  July  31, 2007,  the  Marine  Corps  Systems  Command  launched  MRAP  II  pre¬ 
solicitation,  challenging  bidders  to  develop  a  new  vehicle  that  offered  a  higher  level  of 
protection  than  the  current  MRAP  vehicles.  The  U.S.  Army  Research  Laboratory  ensured 
the  technologies  used  in  the  Frag  Kit  6  (Fullerlove,  2009)  armor  upgrade  project  would  be 
available  to  MRAP  II  designers.  Initial  testing  at  the  Aberdeen  Proving  Grounds  disqualified 
vehicles  that  did  not  meet  requirements;  the  design  run-off  identified  two  vendors  whose 
vehicles  could  pass  the  demonstration  test. 

Information  Technology.  The  federal  and  commercial  markets  have  taken 
advantage  of  the  highly  competitive,  fast-paced  environment  of  IT.  Most  software 
manufacturers  must  prove  that  their  software  works  within  an  environment  and  that  it  can 
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integrate  into  a  larger  system.  Commercial  manufacturers  often  provide  free  demonstrations 
at  trade  shows  and  tabletop  exercises.  To  incorporate  vendor  solutions  into  its  Network 
Integration  Evaluation  (NIE)  program,  the  Army  conducts  semiannual  events  that  bring 
together  three  Army  communities  to  evaluate  militarily  useful  technologies  in  both  laboratory 
and  field  environments.  The  Army  applies  the  Agile  Process  to  accelerate  the  identification, 
testing,  and  fielding  of  relevant  networked  and  non-networked  capabilities  to  the  soldier,  in 
concert  with  capability  set  fielding  and  the  Army  Force  Generation  (ARFORGEN)  cycle. 

The  government  has  also  set  up  programs  specifically  designed  to  make  use  of 
challenges.  In  addition  to  the  Defense  Advanced  Research  Projects  Agency’s  well-known 
Grand  (DARPA,  2004)  and  Urban  (DARPA,  2008)  Challenges,  they  include  the  efforts 
summarized  in  the  following  section. 

Defense  Acquisition  Challenge  (DAC)  Program.  The  DAC  program  (Defense 
Acquisition  Challenge  [DAC]  Program,  2012,  §  2359b)  annually  solicits  technology 
proposals  from  small-  and  medium-sized  enterprises.  The  proposals  present  technologies 
that  could  lead  to  improvements  in  performance,  affordability,  manufacturing,  or  operational 
capability  if  introduced  into  existing  acquisition  programs  (DAC  Program,  2012,  §  2359b). 
The  new  technologies  should  replace  or  augment  some  aspect  of  a  current  procurement 
and  must  be  ready  off  the  shelf.  The  DAC  offers  a  promising  way  to  encourage  innovation 
and  help  new  companies  break  into  the  defense  market.  However,  it  centers  only  on 
improvements  to  existing,  conventional  acquisition  programs.  Ironically,  the  DAC  impels 
these  programs  to  expend  significant  resources  in  order  to  expose  opportunities  for 
innovation  that,  if  successful,  will  render  parts  of  the  original  acquisition  redundant.  In  a 
sense,  the  DAC  represents  an  example  of  ChBA  in  which  the  challenges  are  not  explicitly 
designed  by  the  government  but  inferred  by  industry  from  existing,  specification-based 
acquisitions.  However,  ChBA  has  the  advantage  of  permitting  entirely  fresh  approaches  and 
avoids  forcing  industry  to  accept  the  constraints  of  an  ongoing  acquisition. 

Defense  Innovation  Marketplace.  The  Defense  Innovation  Marketplace  serves  as  a 
centralized  resource  to  help  both  government  and  industry  “reinvigorate  innovation”  and 
fosters  collaboration  and  communication  between  government  and  industry  beyond 
traditional  Requests  for  Information  and  Industry  Days.  The  program  allows  industry  to  learn 
about  the  DoD’s  investment  priorities  and  capability  needs,  and  to  submit  summary  reports 
on  proprietary  Independent  Research  and  Development  (IR&D)  to  potential  customers.  For 
the  government,  the  Defense  Innovation  Marketplace  functions  as  a  one-stop  shop  for  DoD 
science  and  technology  planning,  acquisition  resources,  funding,  and  financial  information 
by  providing  agencies  with  search  tools  to  access  and  leverage  industry  technology  projects 
(DoD,  2013). 

Challenge.gov.  Outside  the  DoD,  the  Office  of  Management  and  Budget  (OMB)  has 
established  the  www.Challenge.Gov  website,  which  helps  individuals  and  companies  to 
compete  for  prizes  offered  by  various  government  agencies  for  solving  some  of  their 
toughest  problems.  The  website  supports  the  “OMB  Guidance  Memo  on  the  Use  of 
Challenges  and  Prizes  to  Promote  Open  Government,”  dated  March  2010.  That 
memorandum  responded  to  the  President’s  Directive  on  Transparency  and  Open 
Government,  which  tasked  the  OMB  Deputy  Director  for  Management  with  issuing  guidance 
for  the  increased  use  of  challenges  and  prizes  to  develop  new  tools  and  approaches  to 
improve  open  government.  OMB  launched  the  website  in  201 1  with  17  different  agencies 
posting  challenges  with  prizes,  including  a  recent  VA  $3  million  prize.  A  progress  report 
published  by  the  White  House  Office  of  Science  and  Technology  stated  that  prizes  may  be 
effective  in  stimulating  solutions  to  government  problems  (White  House  Office  of  Science 
and  Technology  Policy,  2012). 
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ChBA  Attributes  and  Benefits 

ChBA  creates  an  efficient  division  of  labor  where  the  government  focuses  on  what  it 
needs  (i.e.,  demand)  to  achieve  its  mission  and  private  industry  focuses  on  solutions  (i.e., 
supply).  The  government  could  use  ChBA  to  communicate  its  needs  by  framing  challenges 
that  are  analogous  or  identical  to  the  desired  capability.  Industry  could  then  respond  to  the 
challenges  without  being  confined  by  extraneous  constraints  such  as  highly  detailed 
engineering  specifications. 

To  meet  government  needs,  the  challenges  must  be  transparent  and 
understandable.  If  possible,  the  government  should  make  the  challenge  accessible  to  all 
parties  wishing  to  address  the  stated  needs.  Concrete  challenges  can  permit  nuanced  levels 
of  control  in  acquisition  not  possible  with  static  specifications  alone. 

As  shown  in  Table  1,  the  DoD  can  derive  several  benefits  from  applying  ChBA  in  its 
acquisitions.  They  include  expanding  user  involvement,  leveraging  technology,  reducing  risk 
through  proof  of  delivery  rather  than  paper-based  proposals,  accommodating  the  full  life 
cycle  of  a  fielded  system  or  product,  utilizing  the  most  appropriate  contracting  methods,  and 
engaging  industry  to  obtain  competitive  advantage. 
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Table  1,  Acquisition  Considerations,  ChBA  Compatibility,  and  Benefits 


Acquisition  Priority 

ChBA  Attribute 

ChBA  Benefits 

Urgent  Warfighter 
Mission  Needs  / 
Accelerated 

Fielding  Timeline 

ChBA  is  well  suited  to  meeting  urgent 
and  high-priority  requirements.  These 
needs  are  often  very  specific  and 
amenable  to  description  as  acquisition 
challenges.  Additionally,  the  urgency 
of  the  need  relaxes  most  of  the  DoD 
Instruction  5000.02  constraints.  (FAR 
6.302-2  Urgent  and  Compelling  Need). 

ChBA  allows  rapid  development  of 
advanced  technology,  including  both 
military  and  commercial  variants.  It  can 
result  in  fielding  the  correct  solution  the 
first  time,  and  avoiding  additional  costs  of 
rework  and  schedule  slippage — ideal  for 
meeting  urgent  warfighter  needs. 

Technological 

Maturity 

By  definition,  ChBA  requires  vendors 
to  offer  mature  technology  in  order  to 
participate  in  a  challenge  event. 

ChBA  allows  new  functionality  and 
interoperability  to  be  tested  in  a 
concurrent  environment,  ensuring  a  more 
operationally  ready  product  and  thus 
reducing  testing  costs  and  timelines. 

System  Life-Cycle 
Support  /  Upgrade 
Considerations 

ChBA  is  best  suited  for  technology¬ 
intensive  acquisitions,  which  are  likely 
to  be  short  lived  given  the  rapid  pace  of 
technology  evolution. 

ChBA  fits  well  into  short-duration 
programs,  where  constraints  in  the 
Operations  and  Support  phase  of  the 
Defense  Acquisition  Management  System 
process  become  irrelevant. 

Efficient 

Contracting 

Processes 

ChBA  can  be  executed  using  Broad 
Agency  Announcements  (BAAs), 
Indefinite  Delivery  /  Indefinite 

Quantity  (ID/IQ)  contracts,  Single 
Awards,  Blanket  Purchase  Agreements 
(BPAs),  or  Multi- Award  Contracts 
(MACs). 

ChBA  can  employ  a  flexible,  streamlined 
contracting  process  suited  to  a  variety  of 
contracting  vehicle  types.  This  enables  the 
program  manager  to  leverage  the 
contracting  type  that  best  suits  the 
program’s  needs  and  individual  tolerance 
for  risk. 

Enhanced  Industry 
Competition 

ChBA  is  structured  to  encourage  a 
diverse  range  of  industry  members 
(including  nontraditional  defense 
suppliers),  to  participate,  thus  making 
for  a  highly  competitive  environment. 

Because  ChBA  lowers  market  entry 
barriers  to  nontraditional  DoD  suppliers,  it 
provides  enhanced  opportunities  for 
competition  that  may  not  normally  arise 
within  the  traditional  defense  marketplace. 

Law,  Regulations,  Policy,  and  Guidance 

Recent  acquisition  laws,  regulations,  and  policies  emphasize  the  need  to  invest  in 
design  development  and  prototyping  to  mitigate  performance  risk  and  cost  growth  in  DoD 
acquisitions.  In  the  Weapon  Systems  Acquisition  Reform  Act  (WSARA)  of  2009  (Office  of 
the  Secretary  of  Defense,  2009),  Congress  directed  the  Secretary  of  Defense  to  ensure  that 
the  acquisition  strategy  for  each  major  defense  acquisition  program  includes  requirements 
to  demonstrate  capabilities  using  competitive  prototypes,  and  that  programs  consider 
appropriate  trade-offs  among  cost,  schedule,  and  performance  objectives  before 
development  begins. 

Likewise,  the  Federal  Acquisition  System  fully  supports  acquisition  challenges,  as 
indicated  by  the  guiding  principles  in  the  Federal  Acquisition  Regulations  (FAR  1 .102). 
Specifically,  federal  acquisitions  must  satisfy  customer  needs  in  terms  of  cost,  quality,  and 
timeliness  of  the  delivered  product  or  service  by 
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•  maximizing  the  use  of  commercial  products  and  services; 

•  using  contractors  who  have  a  track  record  of  successful  past  performance  or 
who  demonstrate  a  current  superior  ability  to  perform; 

•  promoting  competition; 

•  minimizing  administrative  operating  costs; 

•  conducting  business  with  integrity,  fairness,  and  openness;  and 

•  fulfilling  public  policy  objectives. 

FAR  Part  2.101,  Definitions,  includes  the  following  provision:  “Qualification 
requirement  means  a  Government  requirement  for  testing  or  other  quality  assurance 
demonstration  that  must  be  completed  before  award  of  a  contract.”  The  FAR  and  the 
Defense  Federal  Acquisition  Regulation  Supplement  (DFARS)  contain  regulatory  and  policy 
guidance  to  allow  testing  of  designs  before  implementation  and  fielding.  FAR  1 1 .801 ,  Pre¬ 
award  in-use  evaluation,  states  that  “supplies  may  be  evaluated  under  comparable  in-use 
conditions  without  a  further  test  plan,  provided  offerors  are  so  advised  in  the  solicitation.  The 
results  of  such  tests  or  demonstrations  may  be  used  to  rate  the  proposal,  to  determine 
technical  acceptability,  or  otherwise  to  evaluate  the  proposal.” 

DoD  Directive  5000.01  (DoD,  2003a)  requires  each  military  department  to  establish 
its  own  independent  Operational  Test  Agency  (OTA)  to  plan  and  conduct  operational  tests, 
report  results,  and  provide  evaluations  for  effectiveness  and  suitability.  DoDD  5000.01 
(DoD,  2003a)  further  requires  the  integration  of  test  and  evaluation  throughout  the  defense 
acquisition  process.  DoD  Instruction  5000.02  (DoD,  2008),  issued  in  2008,  requires  a 
Materiel  Development  Decision  prior  to  a  program’s  entry  into  the  acquisition  process, 
causing  program  offices  to  invest  more  funds  to  mitigate  technical  risk.  Such  requirements 
support  the  use  of  ChBA  as  a  means  to  improve  testing  efficiency  and  effectiveness  across 
DoD  OTAs  (DoD,  2003a). 

The  examples  described  previously  show  that  acquisition  law  and  regulation  already 
allow  demonstration  testing  to  ensure  contractor  performance.  Precedents  in  which  the 
government  has  successfully  applied  ChBA  techniques  to  acquisitions  exist  in  several 
domains,  such  as  IT  and  space.  Thus,  applying  ChBA-like  methods  to  satisfy  critical  needs 
appears  both  legal  and  practical. 

An  initial  review  of  acquisition  regulation  and  policy  reveals  when  and  how  ChBA 
may  be  best  applied. 

•  Research  and  development:  A  Broad  Agency  Announcement  (BAA) 
procedure  provides  a  competitive  acquisition  process.  If  the  challenge 
involves  seeking  innovative  solutions,  then  it  almost  certainly  falls  within  the 
area  of  early  exploration  or  development. 

•  Components,  sub-systems,  or  items:  The  smaller  an  acquisition,  the  easier  it 
is  to  adapt  to  the  acquisition  process  without  the  multi-layered  FAR  (2013)  or 
DoD  Instruction  5000.02  (DoD,  2008)  provisions  or  constraints. 

•  Urgent  capability:  Field  commanders  who  require  rapid  action  express  their 
urgent  wartime  needs  in  Joint  Urgent  Operational  Needs  Statements  or 
similar  documents.  These  needs  are  often  very  specific  and  amenable  to 
description  as  acquisition  challenges. 
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•  Short  life  cycle:  Technology-intensive  acquisitions  are  likely  to  be  short  lived 
given  the  rapid  pace  of  technology  evolution.  This  makes  the  complex 
guidance  regarding  the  importance  of  reducing  long  life-cycle  costs  during  the 
Operations  and  Support  phase  of  the  Defense  Acquisition  Management 
process  essentially  irrelevant. 

Better  Buying  Power  2.0 

Recent  DoD  guidance  has  also  emphasized  a  faster  approach  to  adopting  solutions 
by  using  rapid  acquisition  or  agile  techniques.  In  his  “Better  Buying  Power”  memorandum 
(USD[AT&L],  2010),  the  Undersecretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics  recognized  the  need  to  make  DoD  acquisitions  more  affordable  through  added 
investment  at  the  beginning  of  the  acquisition  process  to  ensure  a  cost-competitive  result. 
The  Defense  Better  Buying  Power  (BBP)  2.0  initiative  (USD[AT&L],  2012)  covers  several 
areas  in  which  challenges  can  be  well  suited  to  implement  current  guidance. 

Achieve  Affordable  Programs 

Mandate  Affordability.  Challenges  can  be  used  to  mandate  affordability  by 
requiring  that  all  solutions  meet  a  specific  price  target  as  a  condition  of  participation  in  the 
challenge  and  subsequent  procurement.  For  example,  a  challenge  may  specify  that  the 
chosen  solution  shall  not  cost  more  than  X  dollars.  Challenge  participants  may  automatically 
become  ineligible  for  a  final  contract  award  unless  their  solutions  meet  the  unit  cost  and/or 
total  cost  requirements.  This  approach  ensures  that  all  solutions  that  the  government 
procures  using  ChBA  will  meet  pre-defined  program  affordability  targets. 

Reduce  Program  Cost  and  Risk.  The  government  can  use  challenges  to  reduce 
risk  through  “actual”  demonstrated  performance  before  the  government  commits  itself  to  a 
long-term  contract.  Furthermore,  the  DoD  can  build  testing  and  certification  criteria  into  the 
challenge  event,  thereby  ensuring  that  accepted  solutions  will  meet  testing  requirements 
and  required  performance  objectives  before  they  are  purchased  by  the  government,  thus 
reducing  risk,  delivery  timelines,  and  cost. 

Incentivize  Productivity  and  Innovation  in  Industry 

Incorporate  Innovation  Into  Production  at  a  More  Rapid  Rate.  Challenges  can 
spur  industry  productivity  by  guiding  efficient  application  of  research  and  development 
resources  to  meet  specific  requirements  for  a  concrete  capability.  Furthermore,  because  the 
technology  purchased  must  be  nearly  production  ready  at  the  time  the  challenge  takes 
place,  this  mechanism  provides  an  additional  incentive  for  industry  to  establish  an  efficient 
production  process  that  drives  down  costs  and  promotes  efficiency. 

Promote  Effective  Competition 

Emphasize  Competition  Strategies  and  Create/Maintain  Competitive 
Environments.  ChBA  directly  supports  creation  of  a  competitive  acquisition  environment 
because  it  encourages  a  wide  range  of  solution  providers  to  participate.  Challenges  must  be 
open  to  the  greatest  possible  number  of  potential  participants,  since  traditional  requirements 
for  entering  the  defense  market  do  not  apply  in  the  ChBA  environment.  For  example,  in  a 
challenge  focused  on  current  performance  requirements,  previous  experience  may  be 
irrelevant  when  it  comes  time  to  make  a  contract  award.  This  key  difference  enables 
organizations  and  even  individuals  who  have  little/no  defense  experience  to  participate,  thus 
enlarging  the  competitive  landscape. 

Enforce  Open  Systems  Architectures  and  Manage  Technical  Data  Rights.  The 

DoD  can  also  use  challenges  effectively  to  support  the  introduction  of  open  system 
architectures  (OSAs)  across  the  DoD.  OSAs  require  a  predefined  architecture  with  open 
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interfaces  for  easy  integration  of  components  (DoD,  2011).  Challenges  can  be  used  to 
develop  adaptable  technology  for  key  components  of  open  systems.  ChBA  also  permits 
flexible  intellectual  property  arrangements  and  opportunities  for  licensing  negotiations  that 
support  effective  management  of  technical  data  rights  over  the  program  life  cycle. 

Roles  and  Responsibilities 

Government  Role 

The  government  takes  on  a  new  role  in  ChBA.  In  traditional  acquisition,  the 
government  communicates  its  needs  in  a  specification  and  must  assume  that  fulfillment  of 
the  specification  equates  to  meeting  mission  needs.  However,  the  specification  could  be 
appropriately  constrained,  under  constrained,  over  constrained,  or  simply  wrong.  If  the 
specification  is  under  constrained  or  wrong,  the  result  is  unlikely  to  meet  mission 
requirements.  If  the  specification  is  over  constrained,  the  solution  will  likely  not  be  optimal 
and  might  be  impossible  to  implement. 

Current  incentives  encourage  contractors  to  propose  solutions  to  meet  over¬ 
constrained  specifications,  even  if  the  constraints  create  a  high  risk  of  failure  and,  in  the 
process,  spend  large  amounts  of  money  on  developing  solutions  that  may  never  be  fully 
realized.  The  fundamental  flaw  in  this  process  is  the  failure  to  recognize  when  over- 
specifying  drives  design.  To  avoid  these  problems  and  implement  ChBA  successfully,  the 
government  should  consider  the  following: 

Decompose  Complex  Requirements  Into  Challenges.  The  government  will  need 
to  interpret  warfighter  requirements  and  translate  them  into  meaningful  challenge  events 
that  will  give  industry  the  latitude  for  innovation  and  get  users  what  they  need.  This  requires 
the  government  to  have  a  broad  vision  and  a  commitment  to  success  beyond  that  typically 
needed  to  issue  requests  for  proposals  or  BAAs.  Furthermore,  the  government  should 
ensure  that  technical  details  are  not  over  specified,  but  rather  generalized  into  technology- 
agnostic  capability  requirements  that  can  be  demonstrated  in  a  challenge. 

Generalize  User  Experience  and  Needs  and  Communicate  Them  to  Industry. 

After  gathering  requirements  from  the  warfighter  and  translating  them  into  executable 
challenges,  the  government  should  communicate  the  scope  of  the  challenges  to  industry.  In 
doing  so,  the  government  admittedly  assumes  risk,  because  formulating  the  challenges 
requires  the  ability  to  interpret  and  translate  warfighter  experience  and  needs  in  a  clear  and 
concise  manner,  thus  enabling  industry  to  execute  the  challenge. 

Find  Unclassified  Analogues  to  Classified  Situations.  The  government  should 
employ  ChBA  to  identify  possible  solutions  to  classified  requirements  by  utilizing 
unclassified  challenge  analogues.  In  these  situations,  participants  may  not  know  the  details 
of  the  particular  setting  in  which  the  government  plans  to  use  a  solution,  and  instead  would 
only  know  the  general  performance  objectives  to  be  met.  This  approach  supports  an 
enhanced  competitive  environment  by  enabling  those  vendors  that  do  not  possess  the 
required  security  clearances  and  facilities  to  participate  in  the  challenge. 

Design  and  Execute  a  Concrete  Challenge  Apparatus.  The  government  should 
design  challenge-specific  execution  and  evaluation  processes  that  include  a  plan  for 
communicating  challenges  to  industry,  a  plan  detailing  how  the  challenge  will  be  executed 
contractually,  specific  requirements  for  challenge  participation,  and  detailed  evaluation 
criteria  to  ensure  the  challenge  evaluation  will  be  fair  to  all  participants. 

Perform  Quantitative  and  Qualitative  Analysis  of  Challenge  Results.  The 

government  should  use  quantitative  and  qualitative  measurements  to  evaluate  challenge 
results.  More  specifically,  the  government  may  evaluate  the  challenge  participants  during  or 
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immediately  after  the  challenge,  and/or  over  a  longer  term,  as  defined  by  the  initial 
challenge  notice.  Upon  completion  of  the  challenge,  the  government  may  opt  to 

•  Purchase  one  or  more  of  the  competitors’  offerings  based  on  confidence  in 
the  product’s  utility,  as  demonstrated  during  the  challenge. 

•  Refine  and  reissue  the  challenge  based  on  lessons  learned  during  challenge 
performance.  This  can  become  part  of  an  incremental  government  strategy 
that  includes  challenge-based  research  projects. 

•  Do  nothing.  If  the  challenge  results  did  not  inspire  confidence  that  any  of  the 
products  would  meet  government  needs,  the  government  has  no  obligation  to 
let  a  contract.  This  prevents  a  potentially  unsuccessful  acquisition. 

Industry  Role 

Industry  also  takes  on  a  new  role  in  ChBA:  one  that  more  closely  mirrors  how 
industry  normally  develops  and  brings  a  product  to  the  commercial  market  versus  the 
traditional  defense  acquisition  market.  In  this  case,  industry  would  be  responsible  for 
independently  developing  a  solution  that  addresses  a  given  capability  need  (e.g.,  “get  to  my 
destination  faster  and  with  comfort  and  affordability”).  This  approach  contrasts  starkly  with 
the  traditional  defense  acquisition  process  whereby  the  government  provides  detailed 
specifications  and  requirements  (e.g.,  faster  horses)  to  industry.  In  the  former  case,  industry 
bears  most  of  the  risk,  while  in  the  latter  case  the  risk  is  borne  by  the  government.  Thus,  in 
support  of  ChBA,  industry  should  do  the  following: 

Innovate.  ChBA  will  demand  that  industry  propose  innovative  solutions.  ChBA  is  by 
definition  technology  agnostic — it  does  not  presuppose  one  specific,  ideal  technological 
solution.  Consequently,  government  will  not  prescribe  a  specific  technological  path  that 
industry  must  follow,  but  rather  will  present  its  requirements  in  the  form  of  general  challenge 
objectives  that  must  be  met.  Industry  must  then  apply  its  expertise  to  determine  the  best 
technical  approach  to  address  the  objectives  within  the  schedule/cost  constraints  provided 
by  the  government. 

Cooperate  With  Traditional/Non-Traditional  Entities.  No  single  company  has  a 
monopoly  on  innovative  solutions.  ChBA  acquisition,  by  its  very  definition,  seeks  the  best 
technology  to  address  the  military’s  toughest  problems.  Therefore,  industry  must  be  willing 
to  cooperate  with  any  individual  or  organization  that  could  contribute  to  a  solution  meeting 
challenge  performance  criteria. 

Dedicate  R&D  Funding.  ChBA  will  require  that  industry  dedicate  IR&D  funding  to 
develop  a  solution  that  meets  challenge  performance  criteria.  While  the  government  may 
choose  to  provide  nominal  funding  to  enable  organizations  to  attend  and  participate  in 
challenge  events,  it  may  not  necessarily  fund  any  of  the  initial  development  effort. 

Negotiate  Intellectual  Property  Licenses.  ChBA  will  require  that  industry  be 
prepared  to  negotiate  potential  intellectual  property  licenses  with  the  government.  As  a 
result,  it  is  important  that  industry  properly  identify  which  of  its  solutions  it  derived  through 
exclusive  use  of  IR&D  funding  versus  those  that  may  have  been  developed  at  partial  or  full 
government  expense.  Such  a  distinction  is  important,  because  the  source  of  funding  dictates 
the  type  of  licensing  rights  available  to  the  government. 

ChBA  Within  Defense  Acquisition 

ChBA  is  well  suited  to  smaller  acquisitions,  which  are  usually  not  controlled  by  the 
full  DoD  Instruction  5000.02  (DoD,  2008)  guidance.  In  large  acquisitions,  ChBA  can 
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enhance  the  standard  process  by  efficiently  providing  many  of  the  5000.02-specified 
components,  if  not  necessarily  the  entire  solution. 


Since  ChBA  is  grounded  in  requirements  development  and  the  acquisition  process,  it 
does  not  represent  a  radical  or  disruptive  break  with  accepted  practice.  Instead,  it 
generalizes  and  builds  on  existing  concepts  such  as  the  Defense  Acquisition  Management 
System  (DAMS),  which  guides  the  procurement  of  major  military  systems.  Figure  1  provides 
a  graphical  view  of  the  DAMS  phases. 
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Figure  1.  DAMS  Phases 

The  DAMS  recognizes  the  need  for  an  evolutionary  approach  to  acquisition,  stating 
that  “an  evolutionary  approach  delivers  capability  in  increments,  recognizing,  up  front,  the 
need  for  future  capability  improvements”  (DAU,  2011).  Increments  are  managed  through 
repeated  application  of  the  Technology  Development  and  Engineering  and  Manufacturing 
Development  phases.  ChBA  applies  in  these  early  phases  of  the  DAMS  and  in  the  general 
evolutionary  approach.  Specific  opportunities  for  ChBA  application  within  the  DAMS  are 
further  described  in  Table  2. 
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Table  2. 


DAMS  and  ChBA 


DAMS  Phase 

Applicability  of  ChBA 

Materiel  Solution  Analysis — Assess  potential 
materiel  solutions  and  perform  an  Analysis  of 
Alternatives.  This  phase  begins  when  an  Initial 
Capabilities  Document  is  approved  that  contains 
an  analysis  of  current  mission  performance  and 
potential  concepts  from  across  the  DoD.  It  ends 
when  the  Analysis  of  Alternatives  is  complete  and 
materiel  solution  options,  identified  in  the  Initial 
Capabilities  Document,  are  recommended. 

The  Analysis  of  Alternatives  enumerates  the  critical 
elements  needed  by  each  proposed  materiel  solution. 
ChBA  supplements  this  step  because  industry  provides 
the  technology  needed  to  create  a  capability  prior  to 
participation  in  the  challenge.  If  the  government  does 
become  involved  in  selecting  and  maturing  technologies, 
a  challenge,  based  on  the  needed  capability,  could  be 
used  to  explore  the  range  of  candidate  technologies  and 
assess  their  maturity. 

Technology  Development — Determine  and  mature 
the  appropriate  technologies  needed  for  the  full 
system.  Critical  technology  elements,  identified  in 
the  previous  phase,  must  be  demonstrated  using 
prototypes.  The  Technology  Development  phase 
requires  the  creation  of  a  Technology 

Development  Strategy.  For  an  evolutionary 
acquisition,  the  Technology  Development  Strategy 
is  to  include  a  preliminary  description  of  how  the 
materiel  solution  will  be  divided  into  acquisition 
increments  based  on  mature  technology  and  an 
appropriate  limitation  on  the  number  of  prototype 
units. 

A  ChBA  approach  to  the  Technology  Development 
Strategy  is  to  design  a  challenge  that  proves  the  maturity 
of  each  needed  technology.  The  challenge  may  or  may 
not  require  a  prototype,  but  will  place  emphasis  on 
attainment  of  the  technological  capability  rather  than  the 
delivery  of  a  prototype.  The  acquisition  increment 
requirement  of  the  Technology  Development  Strategy 
can  be  served  by  a  standing  challenge  that  persists 
through  time  as  multiple  challengers  demonstrate  a 
range  of  solutions.  A  standing  challenge  gives  industry  a 
chance  to  improve  on  existing  solutions.  It  also 
encourages  the  discovery  of  game-changing  solutions  to 
challenges  that  have  already  been  solved  with  more 
pedestrian  technologies. 

Engineering  and  Manufacturing  Development — 
Develop  the  full  system  or  some  increment  of  the 
full  system  capability.  This  includes  full  system 
integration  and  creation  of  an  affordable  and 
executable  manufacturing  process. 

ChBA  potentially  eliminates  the  need  for  this  phase 
because  the  technology  needed  to  create  a  capability  is 
already  at  or  near  full  capability  as  a  prerequisite  for 
challenge  participation.  Further,  the  challenge  may 
specifically  require  that  participants  (or  their  partners) 
produce  fully  operational  versions  of  the  submissions  by 
a  certain  point  in  time  following  the  challenge  event. 

Production  and  Deployment — Achieve  an 
operational  capability  that  satisfies  mission  needs. 
This  includes  low  rate  production  for  evaluation  of 
major  systems  and  full  production  or 
procurement  of  smaller  systems. 

Technology  acquired  using  ChBA  is  by  definition  nearly 
production  ready;  therefore,  ChBA  can  be  used  to 
accelerate  the  LRIP  portion  of  the  acquisition  process. 
Furthermore,  if  operational  testing  and  evaluation 
criteria  are  already  built  into  the  challenge  construct, 
technology  will  have  met  T&E  requirements  before  the 
government  makes  a  buy  decision — again  accelerating 
the  IOT&E  part  of  the  acquisition  process. 

Operations  and  Support — Execute  a  support 
program  that  meets  readiness  and  operational 
requirements  and  sustains  the  system,  in  a  cost- 
effective  manner,  over  its  total  life  cycle.  This 
phase  also  includes  disposal  of  the  system  at  the 
end  of  its  life. 

Challenges  can  be  designed  to  ensure  that  operations  and 
support  requirements  are  built  in  from  the  beginning.  As 
such,  a  challenge-based  demonstration  can  reenact 
operational  requirements  for  readiness  and  sustainment 
to  demonstrate  capability  before  the  government  makes 
a  commitment  to  purchase. 

The  ChBA  Process 

Figure  2  shows  the  flow  of  a  hypothetical  challenge-based  acquisition.  The  process 
begins  when  the  government  becomes  aware  of  a  user’s  need.  The  acquiring  agency,  or  its 
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technical  support  organization,  postulates  a  capability  that  can  satisfy  the  user’s  need.  This 
is  a  creative  process  and  requires  more  technical  insight  than  simply  recording  what  the 
user  has  requested. 

With  a  desired  capability  in  mind,  the  government  agency  creates  a  set  of  concrete 
performance  challenges  that  would  demonstrate  the  ability  of  the  envisioned  capability  to 
solve  the  user’s  problem.  For  example,  the  user  problem  could  be  that  soldiers  need  better 
situational  awareness  when  fighting  in  urban  areas.  The  envisioned  capability  could  be  an 
information  sharing  mechanism.  A  supporting  challenge  might  be  to  show  that  solders  who 
use  the  candidate  challenge  solution  earn  better  scores  in  urban  combat  training  than  those 
who  do  not  use  the  solution. 
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Figure  2.  ChBA  Process 

The  challenge  event  can  range  from  large,  periodic,  public  occasions  to  private,  one¬ 
time  visits  to  a  testing  laboratory. 

At  Arrow  C  in  Figure  2,  industry  decides  to  attempt  the  challenge.  This  may  produce 
two  results: 


•  Increased  government  knowledge  of  potential  solutions  and  their  vendors, 
depicted  by  Arrow  D. 

•  Greater  understanding  of  the  trade  space  in  which  a  solution  might  be  found. 
Arrows  E  and  F  show  that  this  understanding  comes  from  both  observed 
performance  in  the  challenge  event  and  information  available  about 
promising  vendors  and  their  products. 

Arrows  G  and  H  show  that  ChBA  can  be  a  cyclic  process. 

•  Competitors  whose  product  failed  in  one  challenge  may  make  another 
attempt  after  modifying  their  products.  The  government  may  also  take  this 
opportunity  to  fund  promising  vendors  directly.  Direct  funding  rewards 
vendors  for  their  initiative  and  incentivizes  them  to  attempt  the  challenge 
again,  as  depicted  by  Arrow  G. 

•  Based  on  improved  knowledge  of  the  needed  capability  and  the  technical 
trade  space,  the  government  can  revise  the  challenge  and  begin  the  process 
again,  as  depicted  by  Arrow  H.  This  can  be  important  during  the  acquisition  of 
complex  systems,  where  multiple  steps  may  be  needed  to  state  the  challenge 
correctly  or  arrive  at  the  appropriate  technology. 
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Case  Study — Joint  IED  Defeat  Organization 

The  mission  of  the  Joint  IED  Defeat  Organization  (JIEDDO,  n.d.-b)  is  to  “reduce  the 
effectiveness  and  lethality  of  lEDs,  to  allow  freedom  of  maneuver  for  joint  forces,  federal 
agencies,  and  partner  nations  in  current  and  future  operating  environments”  (JIEDDO,  n.d.- 
a).  In  its  strategic  plan,  JIEDDO  identifies  as  one  of  its  enduring  capabilities  the  ability  to 
“employ  authorities,  flexible  resources,  streamlined  processes,  and  effective  oversight  to 
drive  the  research  and  development  community  to  rapidly  field  C-IED  solutions”  (JIEDDO, 
n.d.-a).  The  computer  screen  saver  depicted  in  Figure  3  carries  JIEDDO’s  fundamental 
message  to  the  staff  every  day.  This  intensity  of  purpose  and  need  for  rapid  action  make 
JIEDDO  well  suited  to  apply  ChBA. 
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Figure  3.  JIEDDO  Organization-Wide  Computer  Screen  Saver 


In  the  summer  of  201 1 ,  JIEDDO  faced  the  sudden  need  for  a  particular  class  of  robot 
in  the  war  in  Afghanistan.  JIEDDO  demonstrated  strength  and  resolve  by  issuing  concrete 
challenges  that  communicated  the  soldiers’  needs  rather  than  reading  vendor  literature  and 
attending  presentations.  The  challenges  were  drawn  from  the  suite  of  Response  Robot 
Performance  Standards  (National  Institute  of  Standards  and  Technology,  2011)  developed 
by  the  National  Institute  of  Standards  and  Technology  (NIST;  www.nist.gov).  The  NIST  test 
method  suite  includes  a  range  of  mobility  and  duration  assessment  devices  that  provide 
excellent  models  of  the  challenges  robots  face  in  Afghanistan. 


Six  vendors  accepted  the  challenge  and  at  their  own  expense  brought  their  robots  to 
NIST  for  assessment.  Some  robots  met  the  challenges  as  their  vendors  claimed.  Other 
systems  displayed  large  gaps  between  promised  capability  and  demonstrated  performance. 
JIEDDO  then  presented  the  results  of  the  challenges  and  the  concrete  characteristics  of  the 
robots  to  field  users  in  Afghanistan. 


JIEDDO  discovered  that  the  original  request  from  the  field  had  been  over 
constrained.  The  challenge  performance  helped  the  users  to  understand  the  performance 
trade  space  and  to  recognize  that  one  class  of  robot  alone  would  not  meet  their  needs.  As  a 
result,  JIEDDO  identified  two  classes  of  robot  that  addressed  the  concerns  of  two  distinct 
user  communities — an  important  distinction  nowhere  to  be  found  in  the  original  field  request. 


In  addition  to  clarifying  what  the  users  really  needed,  the  challenge  process 
encouraged  vendors  to  improve  their  products  before  the  government  committed  itself  to  a 
purchase.  The  challenge  brought  transparency  and  mutual  vendor  visibility,  sparking 
beneficial  competition  and  product  improvement.  Within  months,  vendors  asked  to  return  to 
the  NIST,  again  at  their  own  expense,  for  another  opportunity  to  confront  the  challenges  and 
improve  JIEDDO’s  perception  of  their  products’  quality.  In  this  way,  ChBA  enabled  JIEDDO 
to  go  from  the  initial  request  for  help  to  fielded  systems  in  less  than  a  year. 
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Implementation 

Barriers 

Barriers  to  implementation  are  rooted  in  the  possibility  that  the  government  will 
attempt  to  manage  ChBA  in  the  same  way  it  manages  a  traditional  technology  acquisition. 
While  ChBA  leverages  the  DAMS  and  supporting  processes,  the  acquisition  pitfalls  that 
plague  these  traditional  systems  could  equally  undermine  ChBA. 


Table  3.  Acquisition  Attributes  and  Implications  for  ChBA 


Typical  Acquisition  Pitfalls 

ChBA  Implications 

Mission  needs  are  incorrectly  translated  through  the 
daisy  chain  of  performance-related  documentation, 
resulting  in  wrongly  defined  system  performance  that 
is  over  specified,  driving  non-optimal  solutions. 

The  DoD  may  not  be  able  to  acquire  the  most 
innovative  solutions  from  industry  using  ChBA  if  the 
government  dictates  specific  requirements  instead  of 
describing  generic  capabilities  to  be  demonstrated  at  a 
challenge  event. 

The  competitive  nature  of  funding  motivates  the 
government  to  make  optimistic  predictions  of  system 
performance  in  order  to  obtain  program  approval. 
Likewise,  industry  is  incentivized  to  propose  risky 
solutions,  since  this  can  lead  to  long-term  lock-in  and 
opportunities  for  contract  modifications  to  address 
product  shortcomings. 

ChBA  fundamentally  does  not  permit  either  the 
government  or  industry  to  over-promise  system 
performance.  Performance  must  be  proven  in  a 
transparent  manner  prior  to  the  buy  decision. 

The  government  often  approaches  acquisition  in  a  risk- 
averse  manner,  requiring  extended  periods  of  risk 
reduction  accompanied  by  documentation  requiring 
multiple  reviews.  Regardless  of  risk-reduction  efforts, 
real  risk  to  the  government  buyer  exists  due  to  the  late 
conduct  of  the  Operational  Evaluation. 

ChBA  addresses  these  risks  up  front  and  is  ideally 
suited  to  high-risk  technological  solutions.  Elowever, 
ChBA  will  require  cooperation  from  current  document 
owners  and  the  Operational  Test  community  to  avoid 
this  pitfall. 

Adopting  ChBA 

In  order  for  the  DoD  to  adopt  and  universally  implement  ChBA  across  the  broader 
defense  enterprise,  we  recommend  that  the  DoD  do  the  following: 

•  Educate  acquisition  professionals  about  ChBA.  There  is  a  gap  between 
the  latitude  allowed  by  current  acquisition  law  and  the  state  of  acquisition 
practice.  Briefly  stated,  the  defense  acquisition  community  culture  tends  to  be 
highly  risk  averse  even  when  there  are  logical  arguments  to  take  on 
additional  risk.  This  cultural  dynamic  is  reinforced  as  program  managers 
regularly  spend  money  to  reduce  uncertainty  (e.g.,  risk;  Frick,  2010,  p.  364). 
ChBA  enables  the  government  to  explore  potentially  high-risk/high-reward 
solutions  in  a  low-risk  environment  before  vast  resources  are  dedicated  to  an 
acquisition  effort.  This  suggests  that  the  acquisition  corps  needs  to  be 
educated  on  the  value  of  using  ChBA  in  these  circumstances. 

•  Publicize  examples  of  ChBA  success.  The  government  should  publicize 
working  examples  of  ChBA  within  the  acquisition  and  supporting  professional 
communities.  Acquisition  professionals  will  feel  more  comfortable  embracing 
ChBA  if  they  can  point  to  other  successful  programs  that  use  ChBA 
strategies.  Senior  leadership  must  be  convinced  of  ChBA  utility  so  that  they 
will  commission  a  few  early  adopter  programs,  and  the  managers  of  these 
early  adopter  programs  must  operate  under  senior  leadership  imprimatur  and 
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protection.  The  success  of  the  early  adopters  will  then  encourage  more 
cautious  program  managers  to  follow  suit,  provided  the  results  of  ChBA  are 
widely  publicized  across  the  DoD. 

•  Develop  a  ChBA  desk  guide  as  a  reference  for  acquisition 
professionals.  The  DoD  should  produce  a  ChBA  desk  guide  to  support  use 
of  ChBA  across  the  defense  enterprise.  The  guide  should  be  patterned  after 
existing  acquisition  desk  guides  to  answer  day-to-day  questions  and  provide 
example  solutions,  practices,  and  business  cases  related  to  ChBA.  As  ChBA 
is  more  widely  adopted  across  the  DoD,  the  desk  guide  should  be  updated 
periodically  to  document  new  lessons  learned,  case  studies,  and  best 
practices. 

•  Consider  legislative  and  regulatory  change.  Amend  the  FAR  and  revise 
current  acquisition  guidance  to  reflect  ChBA  as  an  accepted  method  to 
acquire  capability  for  the  warfighter.  Explicit  acceptance  of  ChBA  in  published 
regulatory  and  policy  documents  will  codify  the  approach  and  bring 
recognition  that  it  represents  a  sound  way  of  doing  business  and  can  achieve 
high  impact  in  performance  improvement. 

Conclusion 

ChBA  can  solve  a  class  of  acquisition  problems  defined  by  industry’s  tolerance  of 
capital  risk  and  the  government’s  ability  to  express  user  needs  in  terms  of  concrete 
challenges.  It  thus  constitutes  a  logical  next  step  in  the  current  wave  of  acquisition  reforms. 
ChBA  has  proven  itself  in  the  world  of  civilian  advanced  technology  acquisition  and  has 
been  demonstrated  successfully  in  limited  areas  within  the  DoD. 

Successful  application  of  ChBA  demands  a  renewed  government  commitment  to 
technical  involvement  in  acquisition,  calling  upon  the  acquisition  agent  to  create  challenges 
that,  if  fulfilled,  would  also  meet  the  user’s  requirements.  This  requires  a  clear  understanding 
of  user  need,  as  well  as  the  creativity,  imagination,  and  technical  insight  necessary  to  design 
the  challenge. 

ChBA  encourages  the  best  performance  in  industry  by  freeing  companies  from 
constraints  unrelated  to  challenge  success.  It  encourages  new  players  to  participate  and 
creates  a  level  playing  field  for  all  involved.  ChBA  adheres  to  government  regulations  and  is 
practical  to  use  within  the  current  federal  acquisition  system.  Above  all,  it  offers  an  efficient 
means  for  stimulating  industrial  innovation  and  reducing  the  time  and  cost  of  government 
acquisition  programs. 
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